Skip to content

[opt](nereids) improve performance of cast big string to complex type#55476

Merged
924060929 merged 2 commits intoapache:masterfrom
924060929:master-opt-cast-complex-type
Sep 1, 2025
Merged

[opt](nereids) improve performance of cast big string to complex type#55476
924060929 merged 2 commits intoapache:masterfrom
924060929:master-opt-cast-complex-type

Conversation

@924060929
Copy link
Contributor

@924060929 924060929 commented Aug 29, 2025 •
edited
Loading

What problem does this PR solve?

improve performance of cast big string to complex type

  1. avoid throw cast exception
  2. avoid compute full hash code of the big string literal
select cast('[1,2,3,4,5,6,7,8,...,99999]' as array<float>);

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@924060929
Copy link
Contributor Author

924060929 commented Aug 29, 2025

run buildall

@Thearas
Copy link
Contributor

Thearas commented Aug 29, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@924060929
Copy link
Contributor Author

924060929 commented Aug 29, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Aug 29, 2025

TPC-H: Total hot run time: 34552 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 69828026a2177df19651bd22d724f59bca1ff432, data reload: false

------ Round 1 ----------------------------------
q1	17616	5420	5172	5172
q2	2029	353	213	213
q3	10208	1306	758	758
q4	10225	1029	561	561
q5	7546	2444	2416	2416
q6	188	167	139	139
q7	949	781	650	650
q8	9362	1341	1138	1138
q9	6956	5133	5187	5133
q10	6987	2441	1987	1987
q11	500	296	271	271
q12	371	355	238	238
q13	17785	3728	3044	3044
q14	254	254	221	221
q15	580	535	482	482
q16	437	428	390	390
q17	610	879	376	376
q18	7539	7296	7169	7169
q19	1087	988	586	586
q20	362	342	235	235
q21	3850	3206	2409	2409
q22	1066	1016	964	964
Total cold run time: 106507 ms
Total hot run time: 34552 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5212	5136	5125	5125
q2	248	330	263	263
q3	2206	2712	2332	2332
q4	1412	1858	1367	1367
q5	4258	4592	4708	4592
q6	231	181	139	139
q7	2038	1981	1840	1840
q8	2685	2720	2586	2586
q9	7425	7419	7423	7419
q10	3181	3326	2928	2928
q11	619	514	495	495
q12	716	782	626	626
q13	3504	3968	3401	3401
q14	304	314	277	277
q15	556	486	521	486
q16	472	514	456	456
q17	1183	1568	1430	1430
q18	7887	7819	7779	7779
q19	845	885	1090	885
q20	2067	2099	2049	2049
q21	5237	4727	4311	4311
q22	1082	1036	1015	1015
Total cold run time: 53368 ms
Total hot run time: 51801 ms

@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 29, 2025

FE UT Coverage Report

Increment line coverage 57.14% (4/7) 🎉
Increment coverage report
Complete coverage report

@doris-robot
Copy link

doris-robot commented Aug 29, 2025

TPC-DS: Total hot run time: 187491 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 69828026a2177df19651bd22d724f59bca1ff432, data reload: false

query1	1065	477	421	421
query2	6617	1760	1776	1760
query3	6754	233	239	233
query4	26174	23868	23339	23339
query5	4384	649	532	532
query6	332	260	240	240
query7	4644	517	313	313
query8	302	260	253	253
query9	8644	2947	2936	2936
query10	517	359	291	291
query11	16043	15053	14873	14873
query12	179	124	126	124
query13	1682	560	433	433
query14	9546	5873	5826	5826
query15	221	188	168	168
query16	7463	683	512	512
query17	1227	752	655	655
query18	2046	455	354	354
query19	216	210	203	203
query20	142	134	125	125
query21	223	133	120	120
query22	4034	4290	4038	4038
query23	34137	32990	33106	32990
query24	8196	2412	2447	2412
query25	627	561	506	506
query26	1253	281	178	178
query27	2716	517	353	353
query28	4372	2284	2261	2261
query29	792	614	501	501
query30	297	228	200	200
query31	913	804	726	726
query32	93	82	81	81
query33	583	402	350	350
query34	798	861	523	523
query35	853	832	771	771
query36	958	1039	914	914
query37	129	120	96	96
query38	4093	4046	4086	4046
query39	1507	1453	1435	1435
query40	229	137	128	128
query41	72	63	62	62
query42	132	124	126	124
query43	590	506	494	494
query44	1341	892	867	867
query45	183	177	177	177
query46	874	1024	652	652
query47	1788	1851	1735	1735
query48	403	420	323	323
query49	741	512	438	438
query50	653	683	410	410
query51	4168	4222	4198	4198
query52	123	117	109	109
query53	262	281	207	207
query54	641	616	542	542
query55	102	95	92	92
query56	355	333	319	319
query57	1202	1225	1141	1141
query58	300	288	295	288
query59	2639	2698	2602	2602
query60	362	347	383	347
query61	185	162	161	161
query62	831	741	690	690
query63	240	192	192	192
query64	4523	1158	852	852
query65	4300	4176	4143	4143
query66	1149	465	353	353
query67	15584	15167	15096	15096
query68	8900	938	589	589
query69	481	390	298	298
query70	1273	1186	1159	1159
query71	469	356	341	341
query72	5994	5136	5277	5136
query73	793	693	371	371
query74	9262	9203	8701	8701
query75	4260	3135	2633	2633
query76	3734	1150	746	746
query77	799	436	347	347
query78	9613	9731	8885	8885
query79	2667	925	610	610
query80	704	599	531	531
query81	497	276	235	235
query82	468	149	116	116
query83	296	272	255	255
query84	303	109	100	100
query85	911	483	425	425
query86	349	317	307	307
query87	4347	4291	4213	4213
query88	2828	2198	2203	2198
query89	449	342	292	292
query90	1968	233	240	233
query91	170	170	137	137
query92	97	78	74	74
query93	1887	998	664	664
query94	700	425	331	331
query95	404	335	326	326
query96	491	614	285	285
query97	2669	2688	2590	2590
query98	252	219	219	219
query99	1469	1450	1310	1310
Total cold run time: 278248 ms
Total hot run time: 187491 ms

@doris-robot
Copy link

doris-robot commented Aug 29, 2025

ClickBench: Total hot run time: 33.21 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 69828026a2177df19651bd22d724f59bca1ff432, data reload: false

query1	0.06	0.05	0.06
query2	0.10	0.06	0.05
query3	0.25	0.09	0.09
query4	1.61	0.11	0.12
query5	0.46	0.43	0.41
query6	1.21	0.64	0.67
query7	0.04	0.03	0.03
query8	0.06	0.05	0.05
query9	0.61	0.53	0.54
query10	0.58	0.58	0.59
query11	0.16	0.11	0.12
query12	0.16	0.13	0.12
query13	0.63	0.63	0.63
query14	0.80	0.88	0.81
query15	0.87	0.85	0.85
query16	0.38	0.40	0.41
query17	1.02	1.06	1.09
query18	0.23	0.20	0.20
query19	1.91	1.91	1.85
query20	0.02	0.02	0.02
query21	15.44	0.96	0.58
query22	0.80	1.27	0.77
query23	14.80	1.39	0.66
query24	6.04	1.64	1.06
query25	0.50	0.27	0.10
query26	0.60	0.17	0.15
query27	0.06	0.05	0.05
query28	9.14	0.95	0.44
query29	12.69	3.92	3.23
query30	3.10	3.05	3.02
query31	2.83	0.59	0.40
query32	3.24	0.57	0.48
query33	3.12	3.10	3.07
query34	16.12	5.44	4.82
query35	4.93	4.92	4.96
query36	0.69	0.51	0.49
query37	0.10	0.08	0.08
query38	0.06	0.04	0.04
query39	0.04	0.03	0.03
query40	0.17	0.15	0.14
query41	0.09	0.04	0.03
query42	0.04	0.03	0.03
query43	0.04	0.03	0.04
Total cold run time: 105.8 s
Total hot run time: 33.21 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 29, 2025

FE Regression Coverage Report

Increment line coverage 85.71% (6/7) 🎉
Increment coverage report
Complete coverage report

@github-actions
Copy link
Contributor

github-actions bot commented Sep 1, 2025

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Sep 1, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Sep 1, 2025

PR approved by anyone and no changes requested.

@924060929 924060929 merged commit d44fee0 into apache:master Sep 1, 2025
28 of 30 checks passed
@924060929 924060929 deleted the master-opt-cast-complex-type branch September 1, 2025 06:17
github-actions bot pushed a commit that referenced this pull request Sep 1, 2025
@924060929
[opt](nereids) improve performance of cast big string to complex type (…
855def5 
…#55476)

improve performance of cast big string to complex type
1. avoid throw cast exception
2. avoid compute full hash code of the big string literal


```sql
select cast('[1,2,3,4,5,6,7,8,...,99999]' as array<float>);
```
@github-actions github-actions bot mentioned this pull request Sep 1, 2025
Merged
morrySnow pushed a commit that referenced this pull request Sep 4, 2025
@github-actions @924060929
branch-3.1: [opt](nereids) improve performance of cast big string to …
758fce9 
…complex type #55476 (#55521)

Cherry-picked from #55476

Co-authored-by: 924060929 <lanhuajian@selectdb.com>
@morrySnow morrySnow mentioned this pull request Sep 22, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@starocean999 starocean999 starocean999 approved these changes

@morrySnow morrySnow morrySnow approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/3.1.1-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@924060929 @Thearas @doris-robot @hello-stephen @starocean999 @morrySnow